This specification relates to dialogue systems.
A dialogue system is a computer system that has conversations with users by generating system outputs in response to user conversational turns.
For example, during a given conversation, a dialogue system can receive a speech input from the user that represents a conversational turn, convert the speech input to text, and then operate on the text to generate a speech output that is a response to the speech input received from the user.
Dialogue systems can be used to converse with users to accomplish any of a variety of tasks. For example, a dialogue system can be used to allow users to select options from a menu, e.g., food items from a restaurant menu. As another example, a dialogue system can be used to allow users to make a reservation, e.g., a restaurant reservation or a travel reservation.
Some dialogue systems use neural networks as part of generating a system output from a user input.
Neural networks are machine learning models that employ one or more layers of nonlinear units to predict an output for a received input. Some neural networks include one or more hidden layers in addition to an output layer. The output of each hidden layer is used as input to the next layer in the network, i.e., the next hidden layer or the output layer. Each layer of the network generates an output from a received input in accordance with current values of a respective set of parameters.
Some neural networks are recurrent neural networks. A recurrent neural network is a neural network that receives an input sequence and generates an output sequence from the input sequence. In particular, a recurrent neural network can use some or all of the internal state of the network from a previous time step in computing an output at a current time step.
An example of a recurrent neural network is a Long Short-Term Memory (LSTM) neural network that includes one or more LSTM memory blocks. Each LSTM memory block can include one or more cells that each include an input gate, a forget gate, and an output gate that allow the cell to store previous states for the cell, e.g., for use in generating a current activation or to be provided to other components of the LSTM neural network.